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Introduction 



Recent advances in pattern classification have enabled the development of sophisticated software 
systems that can recognize natural language input such as speech [6] or handwriting [71. These 
apphcations allow users to communicate with the system in a natural and convenient way, and 
permit die automation of tasks that previously required human input Some examples of such 
actions include interactive voice response (TVR) systems, automated chequlprocessing 
systems, and automated form data-entry systems. 

In addition^* growth of networked computing and the Internet has enabled the development of 
2™*o^ Sy f mS ' "* CXistence of °P en - standardized protocols has allowed the 
2S °^^JrT^' C T^ SefVerS ' md a PP"<*tions. An example of a three-tiered 
dislnbuted system architecture is depicted in Figure I. The combination ofdistributed computing 

ul^m reCOg ? Uon . tecbniclu ? made Possible the development of systems such as 
Netpage [1] an interactive paper-based interface to online information. Systems such as these 
grve users the ability to interact with their information from any location tfcu^vlSes nctwS 

sssaassisr netwoik — * — — jss 

This paper discusses some of the technical issues involved with integrating pattern recognition 
T a .f tribUted ^ P^ses a generic Sneworic £ pTrSung 

m£ ?J? ^gmtion usmg centralized recognition servers and distributed context processing 
Also discussed are techniques for managing the user-specific customisation and adaptation that I 
required to make pattern recognition systems accurate and flexible. 
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1.2 Pattern Recognition 

Lt'TiT 88 '" 8 . ° f 8 pattem reco g nition ^tem are depicted in Figure 3. Processing 

pTe^ 61131 SC8mentation ^ resolved during classification w context 

eJtractTS*^ rtf.T' ¥ *T * 8 C,aSsifier where a «PresentaUve set of features is 

SC * IZiSlS? h ^ m JS° mb r « on *** «» pre-defined model of the input s^a! to 
sZrace *tf£S?S? J^T^^T hyP ° the ? eS an indication of the probabUity^iat a 
sequence of segments within the signal represent a basic symbolic element fe e. letter word 

Sbt^ C fi b fft f ^^rocessing -Tdule ustTsSL SoS 
Sat or SSS 6 " ° ^f^* 0 *"°"««e to a specified context model Such a^ 

fets;^ produced by 1,16 context processbg is paid to * e 
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Context Processing 

applicJonstoopcrate^Sssfi^v ^S^^ 18 for P attem cognition 

result, recognition sysi™ mm i mS? ,~ ^ ^ "if CT P ensive ^ frustrating to users. As a 
toreLtheTossft^ information as possible to 

mat must represent a co^^e^f^LTf 5" ^ reco 8^8 * signal 

country names to guide thTre^ti™ ^ocphT « ^ ?° n " c a P""**"* list of valid 
^tedsynfcolset^^ 

information from a Srent soS 'aTT* "T be accurate, y classi fi*d without further 
development of WgMy a™te 2± ° ne of mc ^ raced in the 

handwriting <■* * the inherent ambiguity of 

similar appearance and are thus eSly conW^ Hunt, V , m ^ Written ^ a very 

correctly decode handwritten tex? and ^^^T^ ^ rely ° n contextual knowledge to 
applying syntactic mTlZZSc ^in^ ,, ^™»* of research has been directed at 
Similar work has beTperfSmed ui t^eTS S^T T*™ reCOgnitio11 [8,9,10,11.12,13]. 
and machine translation: Speecb recognraon, natural language processing, 



Current Systems 



uST^baid 51 ;: *TS!S52^ rccognition system caUed Ne ^P3 

handwriting -cuSi^^^^ natural 
then transmitted *\^5?^ by com P act clicnt software > 

and returned as typewnSnt^^^ t0 NetC ^ servers where it is converted 

recognition, and correction cSe do^T/Sf^ vocabularies (called ConTexts) and "dictation, 
the totemcf' [3]. C ° mCtl0a be done > ^ependently of the location, across a LAN, WAN, or 

Detailed Description of the Preferred Embodiments 

including in particular PCT a PP UcaSrwOoS5989 entiZ £ OUr applications, 
2002, PCT application WO0242894 S !? 'Sensmg Device" filed 30 May 

appUcation W^0214^ «Tnt^r« ^- Interact,ve Pri nter» filed 30 May 2002, PCT 
PCT vS^SSmo^^J"^ ***** MC " ffled 21 February'200K 
filed 30May 2002^^^ CompuSrSystem" 

Using Handwriting Feature ^SSS^Su^SS^Sf^ ** J^* 56 Searchme 
implementation will necessarily embody all or tS™^?*^ WfHated that not every 
described in these applications in Nation to Z w . ° f J" specific details extensions 
its most complete fom to^sSf uS^J? T SyStem - . However . *e system is described in 
and aspects of the ^SSL^SS^* " **** *° preWd -bodimen te 

^T2etes trjssssr a r era ? *— *• • 

imaging pen. Each pageTenerated blftSLST °! COded pa P er 3,141 "> 

network server, and aS u?eT£ttra^tten ^SS^ST " Mentified «* storcd °* a 

interpreted, and stored. DiS U%£S * .«« * -f^, 

documents, allowing interactive apphcations to^e ndJ3S5 "^^P^g ^Netpage 
network infrastructure provide a paper-based SamnvTSSil JSf , Net P a 8 e P"nter, pen, and 
and online publkhmg%ervices, ^d supports ^rZ^T^ 180 ^^^ 1 ^ 0 ^ 
navigation and form hjut sports user-interface fiinchoaality such as hypertext 



Typically, a printer receives a document from a publisher or application provider via a broadband 
connection, which is printed with an invisible pattern of infrared tags that each encodes the 
location of the tag on the page and a unique page identifier. As a user writes on the page the 
imaging pen decodes these tags and converts the motion of the pen into digital ink. The digital ink 
is transmitted over a wireless channel to a relay base station, and then sent to the network for 
processmgand storage. The system uses a stored description of the page to interpret the digital 
ink, and performs the requested actions by interacting with an application. 

Applications provide content to the user by publishing documents, and process the digital ink 
interactions submitted by the user. Typically, an application generates one or more interactive 
pages in response to user input, which are transmitted to the network to be stored, rendered, and 
finally printed as ou»ut to the user. The Netpage system allows sophisticated applications to be 
developed by providing services for document publishing, rendering, and delivery, authenticated 
transactions and secure payments, handwriting recognition and digital ink searching, and user 
validation using biometnc techniques such as signature verification 



Distributed Pattern Recognition 

recognition, with the m^t^SZJtt^TS^ ° W IT™* to a for 
intopretationand processing. OnealZIrfftl ^ u a *"* application for 
application systerns do 2 hE si^™?*^ 18 ^ client ^vices and distributed 
perform pattern recognition, aXe IKlS? COmm0nly needed to 

to the resource constraints flat aTmEI^T perform foe recognition are not subject 
digital assistants, imaging pens Tc ) ^T^T uf^* (C fr m ° bile P hones ' P^°nal- 
memory-intensive techdquesto ^Si^ aWe to V s6 processor- and/or 

perform the specific recognition X reCOgmtIon acourac y> and «se naxdware optimised to 

Performing pattern recognition on a centralized server ako „<w 

recognition systems that employ user-specific TdTnSTJ ll- u- f advsmt age to Pattern- 
example, some handwriting re »S£S 2 reco « nition rates. For 

the system based on preWoTSoS£?3^S 1*7*™*"* model for each — of 
accuracy of the system for that user W T S h^hL ™t to ^ *• &ture 

performs all recognition for a user [rafoer thT ? T * a ^ ««"*■ 

performing their own cognition), S foe^erTaoTton^ ^ *™ Uc *«<™ <*<* 
mput generated by aU applications In addiZ.^^ _,. t ° Perform adaptation based on the 
simplifies the management o^TrSo^tion 2£S abased pattern recognition 

and upgraded without interaction wlfo E dSZSc^ ^ re ? 0gnizers to be reconfigured 
and test data to be easily collected. cbents md applications, and allows training 

However, the information reauired to ™^-fx, 

recognition system is generaUy^caL^Sc a^d T*, ~" ° f 3 P 8ttera 

application-specific database) mfSdnTS sE25 and is often very large (e.g. entries in a large 
centralized sender forprocSg Sutio^oS m ** COntext information to a 

recognition as depict inl?^ ' S ^° a ^P roblem ^ to ^ e « meclmism for distributed 
processed by an ^plication, thT^l^ u b ^S SAT* 1 t0 1° -d 

server performs pre-processing, segm^taltaT^ for Prncessing. The 

to decode the result (or only performs ZISS S^ does not 1186 a context model 
Rather, the intermediate reiogS Stf^L m foUowin S Hussion). 

aPPKcationtoapplyanyafoitraX^^^ apph ' Cation allowing me 

signal -""y complex and domam-specific context processing to decode the 



2.1 Symbol DAG 



£o"5£5 re aS 8 g^pS te ^TSeri: ^ 10 " — 
associated scores JveruSnd vSnLtioSLt ^T* *" COTtaias *»**«id 
implemented as a two-dunensionJ ^^Sem^. u 3 83 edgcs - ^ structure can be 
by the pattern classifier for ? Selegment SiS ^t"* *» ° Utout grated 

that segment. This structure repWne aU ^ 

derived from the input signal 2ST£ foe tsuKt ^^f 011 alternatives that may be 
structure, in combiXn wSfa coTtex^^ol t t^S^ ^ — 

probability mat the clLifieSegmVnt is Scute sv2£l "JT* C ° lumn repreSents 
the next possible segment (column "fa foeTom S t"" 1 T» dCS M ofi&et 1,181 mdic «es 
matrix represents all the possible dec^dW oafol K ^ ^ foll ° W to Symbo1 - Thus > «he 
These paths and associated ^S£S^t^J^^ OUtou u t of *» Pattern classifier, 
decode the input signal. ,assmca «on scores can be combined with a context model to folly 
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symbol DAG may be any primitive ZilrnZt of constraints. The symbols contained in the 

including the ^ut ^^^Z^lr ^T S °l* ^ 
handwriting and sneech remcmi^n ' Slfaer Samples of such recognition systems include 

vision [San7ec^mcSLT6] * ™' ^ piOCeSSin8 "* 

Symbol DAG Example 

c^cteT^f geLrtrbv T^o*^™* ? ° UtpUt *° m * ««en 
recognizer has found tw? possible ctrSrT g» ven m R»« 4. In this example, the 

rows fothe symboT DAG Note ffi L^ «S™»«"» arrangements, as depicted by the two 
however, anSua hnpL«Ldo^ Zt^T^' ?" "ti ^ 316 38 P">°abilities; 
of the p^abih^ft^Sove^ ^P™^** 0* base-10 logarithm 

and underflow plS lSS? 3S3SZ° f ^^P™?^ «4 to avoid overflow 
point operations. multiplying probabilraes using finite precision floating- 

c^^c^^n^ *f. COntext P roc — starts with the first entry in the DAG (i e the 

■ h i 5 a I ^sr^SoS i 's^roi , Spi;r: ,b ' 

clog » 0.7 * o.8 * i.o * i.o « 0.56 
<*g - 0.7 * 0.2 * 1.0 „ 0.14 

dog = 0.3 * 1.0 * 1.0 „ o.30 

"SSSZSLS ^. C ° mbined ^ a lan ^ e -del or other contextual information to 
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Table 1. Example DAG for «clog»/»dog" Ambiguity 

letter may use font stroke m iHa^S? a„ t ? ? f 531 ^ to « l<*er, no subsequent 
derived DAG depicted b Table aTSf^l? i amP ,. le """f fe ^ m Fi »» 5, with the 
recognized as crowbar elements of ? iSLTSJ. ?" J"?* h ° rizontal «»»** <*« PotentiaUy be 
marking is used as a crSr it 125 JLi, d,ac " tl0 u aI ** letter «i\ However, if a 
decoding mStivL m Ze^W ■*"*"'* be used as a diacritical. The potential 



tile - 0.6 . 1.0 . o.S * l.o - 0.36 

tite - o.e • i.o . i. 0 , x>0 „ 0 60 
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lite - 0.4 * l.o * 1.0 ♦ i.o = 0.40 

These values can now combined with a language model to select the most likely word. 





0 


1 


2 


3 


4 


5 


Character 

Offset 

Score 


t 
1 

0.6 


i 
4 

1.0 


i 
3 

1.0 


t 
5 

1.0 


1 

5 

0.6 


e 

1.0 


Character 

Oflfeet 

Score 


1 

2 

0.4 




t 
5 

0.4 





Table 2. Example DAG for "fite'VW Ambiguity 
certain portability ^ a ^ ^ at *™ * • 
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Table 3. Example DAG for SPACE/NUL Pair 



potential decoding alternatives in mis example 

ab = 1.0 ♦ 0.6 ♦ 1.0 = 0.6 
a b - l.o * 0.4 ♦ l.o - 0.4 



2.3 Distributed Recognizer Management 

IJSSSt rS£L Sy ^T y SUPP ° rt 3 DUmber of recognizer, that are 

systems capable ^portinTdSt ^ ^ can ^ 

dialects, or accents, or S oJboS S ^T^ 8UCh 83 dfflto " t 
requires a recognition task t^e perfomeTh ™? ^ * 8 ^ 3,1 a PP**«<» 

recognizer that matches the panunS^ tt • firs ^ t l u lk enes *° ^cognition manager to find a 
The recognition tnana^tneT^L L^S W ■ be rcCOgnized ^ in Figure 7). 

parameteTspcSn^y^e applSn ShtTSJ° V * *" Bm *" 8Upp0rts * e 

*~inceme^^^^ 
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2.4 User-Specific Dictionaries 



Distributed recognition systems can also support user dictionaries, which are user-specific word 
lists (and possibly associated a-priori probabilities) that include words that a user writes 
frequently but which are unlikely to appear in a standard dictionary (examples include company 
names, work or personal interest specific terms, etc.). User dictionaries can be stored and 
managed centrally so that words added to the dictionary when using one application are available 
to all applications for context processing. Obviously, applications can manage and use their own 
local user-specific dictionaries if required, since they have full control over context decoding. 

When an application requires the recognition of a signal that may contain words found in the user 
dictionary (e.g. standard handwritten text input such as the subject line of an e-mail or an arbitrary 
voice message), the centralized recognition system generates the usual intermediate recognition 
results to be returned to the application for context decoding. However, in addition to this it 
decodes the intermediate results using the user-dictionary as a language model, the result of which 
is also returned to the application. These two intermediate results structures can be combined by 
the application during its context decoding to generate a final decoding that includes the user- 
specific dictionary information. 

2.5 User-Specific Training 

Distributed recognition systems may also support user-specific training for recognizers, as 
depicted in Figure 8. The data generated by a user-specific recognition training application is 
submitted to the centralized recognition manager, which stores the data in a database. The 
recognition manager then enumerates all recognizers to determine if they support the data format 
as defined by the parameters associated with the training data, and if so, submits the training data 
to the recognizer for user-specific training. 

When an existing recognizer is upgraded or a new recognizer is added to the system, the 
recognition manager queries the training database to determine if any training data of the format 
required by the recognizer exists. If so, the training data is submitted to the newly registered 
recognizer for processing, as depicted in Figure 9. 

3 Conclusion 

A number of techniques that allow pattern recognition to be performed in a distributed system 
have been discussed, including a method of returning the intermediate results generated by a 
pattern classifier to an application for context processing. In addition to this, techniques for 
managing multiple recognizers, user-specific dictionaries, and the user-specific training of 
recognizers have been given. 
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Figures 
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Figure 1. Distributed System Architecture 
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Figure 2. Distributed Recognition 
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Figure 3. Basic Pattern Recognition 




Figure 4. Ambiguous Input Ink for "clogT'dog" 




Figure 5. Ambiguous Input Ink for "tile'VUtc" 



NPW013 



Input Device 



Manager 



I Supports (Parameters) 



Recognizer 



Signal 



Get Recognize r (Parameters) 



True 



Application Response 



Application 



Recognizer 



Rec ognize (Signal, Param eters) 



Symbol Lattice 



Application requests a 
recognizer that supports 
the specified parameters 



Application uses the 
context to decode the 
symbol lattice 



Figure 6. Example Recognition Scenario 
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Figure 7. Recognizer Selection Scenario 
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Figure 8. Recognizer Training Scenario 
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Figure 9- Recognizer Registration Scenario 
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